In a prior U.S. patent application entitled xe2x80x9cSUBCHANNEL MODULATION SCHEME FOR CARRYING MANAGEMENT AND CONTROL DATA OUTSIDE THE REGULAR DATA CHANNELxe2x80x9d, filed Apr. 20, 1998, Ser. No. 09/063,633, the teachings of which are hereby incorporated by reference the details of a subchannel transmitter were taught. This application teaches how to use such a subchannel transmitter to implement network management out of band but sharing the same media as the high speed data.
The invention is useful in Fibre Channel Arbitrated Loop (hereafter FCAL) or other computer network or telephony network management systems to, in the preferred embodiment, reduce the cost of ownership and provide a simple, lightweight, out-of-band (or in-band) network management protocol which is SNMP compatible and which has integrated, unilateral device/topology discovery using a synchronous command protocol.
FCAL is a form of computer network which is popular in high speed, high volume networks. The networks are comprised of one or more hubs and a plurality of nodes such as computers, printers, disk drive arrays etc. Like all computer networks, the nodes on the network are coupled together by a data path and the number and arrangement of nodes on the network can change over time.
Computer networks must be managed. Data must be obtained from the various nodes such as traffic levels, temperature etc. and various conditions must be set in the network to open ports, shut ports down etc. The various registers, ports, sensors etc. in a network are the objects to be managed. Typical network objects are enclosure temperature, port status (fault, bypass etc.), power supply status, fan status, traffic level etc. The temperature and fan status are important objects in high speed hubs since the hub will fail shortly after its temperature gets too high or its fan stops working.
SNMP, or Synchronous Network Management Protocol is an industry standard protocol enabling management of networks. SNMP protocol provides an industry standard way of performing get and set operations. Each of the objects to be managed in the various nodes has an object ID. Get operations cause data to be obtained from the object identified in the object ID (hereafter OID) in the get command. Set operations cause data to be written to the object defined by the object ID given in the set command.
Network management station (hereafter NMS) software that provides an interface to a network management employee and allows him or her to manage the network by issuing get and set commands is known in the art. There are many software programs available commercially that provide SNMP compatible network management services. These programs reside on a computer coupled to the network and can either communicate with the nodes in the network directly or through so-called proxy agents. Originally NMS SNMP software was designed to talk to a single device on the network. As networks grew in size, various solutions to this problem surfaced. Proxy agents were one of these solutions. An example of a prior art NMS software package is Open View(trademark).
Proxy agents are known and are a way of reducing the amount of software and licenses that are needed for the nodes on the system. If a network is arranged so that NMS SNMP management software manages the network by communicating directly with the nodes in the network, more software are needed than if a proxy agent is used. NMS software typically communicates with the objects in the network by encapsulating SNMP packets inside Ethernet packets and sending the Ethernet packets to the nodes on the network which contain the objects from which data is needed or to which data is to be written. NMS software can use other protocols than SNMP and Ethernet. To manage a network in this way requires that each node have an Ethernet interface card and a software driver for that card plus UDP/IP layer software to understand the IP packet address header and SNMP layer software to understand the get and set commands, decode the OIDs and carry out the desired functions. This requires an SNMP license for every node plus the expense of the UDP/IP layer plus the expense of the Ethernet software and hardware at the physical layer. Proxy agents cut down on this expense by incorporating the Ethernet driver and interface card and the UDP/IP and SNMP layer software therein to receive SNMP commands from the NMS software. The proxy agent then uses its own less complicated communication protocol to communicate with the other nodes in the system in a master-slave relationship to carry out the get and set commands. An example of a prior art proxy agent software program is the proxy agent available commercially from ISI or the VX Works(trademark) package available commercially from Wind Works.
A MIB is an abstract definition document containing data that is used by the NMS software and the proxy agent to learn the network objects available in a particular network that can be managed. The function of the MIB is to define the objects in the nodes that are available for management (sometimes called xe2x80x9cexposingxe2x80x9d them) to the NMS software and the proxy agent. MIBs are written in custom fashion for each network to define the objects in the network to be served by the hubs in the network to the NMS software. The MIB defines the syntax that the NMS software should use to ask for data from an object or set data into the object. The MIB defines the OIDs of the objects in the network so that the NMS can do gets and sets to these OIDs and so that the hubs know which objects to which the gets and sets are directed. The MIB also defines what type of data, e.g., floating point, 4-byte integer etc., is output by an object or what type of data to write to an object. The hubs in the network serve as the servers which serve the objects in the network to the NMS software. Every hub has a MIB which lists the objects which can be viewed and controlled by the NMS software process.
In a broadly defined embodiment of the invention, several elements described below are combined to provide a system for reducing the cost of network managment by using a proxy agent and subchannel communications sharing the same media as the high speed data of the network. For purposes of clarity in separating the notions of the mainstream data traffic on the network and the management data and commands, the phrase xe2x80x9chigh speed dataxe2x80x9d will be used to denote the mainstream data signals which are not management data and commands. However, the phrase xe2x80x9chigh speed dataxe2x80x9d in this specification and claims should not be understood as limiting application of the invention to networks where actual high speed data such as 1 GB Ethernet or FCAL is being transferred. The various embodiments of the invention are useful in any network comprised of a plurality of objects to be monitored and/or managed regardless of the speed of the mainstream data traffic.
By using a proxy agent and subchannel communications, the additional expense devoted to network management is reduced because fewer SNMP licenses and fewer Ethernet and UDP/TCP protocol stacks are needed and no separate wiring for management communications to each node is needed. Further, because the subchannel data shares the media used by the high speed data and there is no separate Ethernet or other LAN coupling each node and carrying solely managment traffic, there is also no need for an Ethernet or other interface circuit at each node.
Although the preferred embodiment described herein uses a subchannel out-of-band communication path, in alternative embodiments, the management protocol can be practiced in-band also. These embodiments do not require the mapping process and routing instructions in each subchannel packet but require management data encapsulated in regular packets normally used to tranfer high speed data and require a process to crack open each data packet in the mainstream, look for management data, route the packet or repeat the packet onto the segment containing the target node and management object and a process to crack open the packet at the target node and act on the mangement request.
Subchannel communication is achieved in a plurality of different embodiments. Embodiments having single subchannel transceivers, multiple transceivers, single multiplexer and multiple multiplexers are disclosed.
An NMS process using routing table cyclic redundance check calculation (hereafter CRC) to automatically detect when the NMS topology information is incorrect and automated topology discovery is disclosed. A process for automated discovery of redundant cables during automated topology discovery is also disclosed.
In a broad embodiment, the subchannel communication protocol and apparatus stands alone regardless of what type of NMS or other process supplies the object IDs and desired operations to be performed on those objects. In this broad embodiment,(a proxy agent receives a management command identifying the object upon which a management function is to be performed such as writing data to the object or reading data therefrom. The proxy agent uses the object ID as a search key to look up a function call in a mapping table which maps object IDs to specific function calls associated with a computer program in a library of programs that has an application programmatic interface. Each program is invoked by a specific function call of the API and controls the hub or switch microprocessor to do all processing necessary to carry out the desired management function on the object identified in the management command. The program is executed, and, if necessary, it builds a subchannel packet using the object ID in the management command and routing instructions read from a routing table which stores routing information based upon the current network topology to get a subchannel packet to every object ID in the network. The particular communication protocol to get the subchannel packets to their target network objects is not believed to be critical to the invention. For example, in some embodiments, the routing table can be eliminated and each subchannel packet contains at least the object ID and information codifying the management function to be performed. Each subchannel packet in this embodiment is then broadcast on the subchannel to all nodes. The particular switching protocol for this embodiment for the subchannel transceivers depends upon the network topology. Each node looks at each subchannel packet and decides whether the packet is addressed to it or not. In the preferred embodiment, the nodes in the network are arranged in a heirarchical structure, each node except the master node having exactly one parent node and possibly having one or more child nodes. Each node has a subchannel transceiver which is controlled to constantly monitor the uplink port coupled to the parent to receive subchannel packets and uses routing instructions in said subchannel packet to switch its transmitter to output the subchannel packet on the correct downlink port to send the subchannel packet on its way to the target object. In each node, after transmitting the subchannel packet out a downlink port, the transceiver receiver is switched over to constantly monitor the downlink port for any reply packet or message transmitted by the node having the target object, and when such a subchannel packet is received, the transceiver sends it out the uplink port to its parent node, and then switches its receiver to begin monitoring the uplink port again for further subchannel packets or topology discovery messages.
The first element in some embodiments is a network management process having any type of user interface such as a graphical or non graphical interface which functions to send management packets to a proxy agent to obtain data and set various network objects to desired values. The network management process can use any type of management protocol and any type of database which exposes to it the objects on the network available for managing. Preferably, SNMP protocol and a MIB are used. In the claims, the phrase xe2x80x9cnetwork management processxe2x80x9d is used to refer to any type of NMS process using any type of management protocol and any type of database to expose the objects to be managed.
Transmission of the management packets to the proxy agent can be by any method such as over any type of data network or over an interprocess transfer mechanism. In the claims, the phrase xe2x80x9cdata transport mechanismxe2x80x9d is used to refer to whatever mechanism is used to get the management packets to the proxy agent regardless of whether the network management process and the proxy agent are running on different computers at the same location, at different locations or on both running on the same computer.
The proxy agent process can either cause transfer of each management packet to the object which is the target thereof by an out-of-band subchannel transmission or can cause the management packet to be converted to a packet of another format. The packet of another format or the management packet itself transmitted over the subchannel is referred to in the claims as a subchannel packet. If the management packet has been converted by the proxy agent into another format, the other format can be any necessary format to aid in implementing the communication protocol used to transmit the subchannel packet to the object such as the GUMP packet format described in the detailed description.
The transmission of the subchannel packet to the network object to be managed may be by any communication protocol, so long as the transmission is made over the same media as the high speed data and is made out-of-band. xe2x80x9cout-of-bandxe2x80x9d means the subchannel packet is not encapsulated in any packet of the type used to transmit the high speed data and does not reduce the high speed data throughput. In the broadest definition of the invention, the type of communication protocol to get the management packets from the proxy agent to target network objects is not important, and in the claims, the communication protocol from the proxy agent process to the objects to be managed is simply referred to as xe2x80x9cany subchannel communication protocolxe2x80x9d.
The subchannel transmissions are made by one or more subchannel transceivers in each hub and node on the network. The number of transceivers used to do the subchannel communication protocol is also not important. The hubs and nodes usually have multiple ports, and if the number of subchannel transceivers is less than the number of ports, one or more multiplexers are used to couple the transceivers to the ports. The number of multiplexers used is in accordance with the subchannel communication protocol and is not critical to the invention in the broadest definition thereof.
The type of high speed data network carrying payload data is also unimportant so long as a type of encoding is used on the payload data which is DC balanced or otherwise encoded so that a portion of the Fourier spectrum of the payload data has suppressed energy levels and can be used for subchannel communication of overhead data such as subchannel packets with minimal interference from the high speed data. The phrase xe2x80x9cdata networkxe2x80x9d in the claims is intended to mean any type of data network as defined in the previous sentence.
More generally stated, the main genus of the invention includes all species which solve one main problem. This problem is how to reduce the cost of ownership of a network wherein both high-speed, high-volume data and low-speed, low-volume management data must be sent without having an adverse impact on the throughput of the high speed data. This problem has three cost components: the cost of a separate data path for management packets to every hub or node having objects therein to be managed; and, the cost of a separate protocol stack at every hub or node, the protocol stack being those software processes to process the overhead aspects of transmitting the management packets to the devices to be managed; and the cost to implement a communication protocol between the master hub and proxy agent process and the slave nodes. Specifically, if an SNMP management protocol is used and the SNMP packets need to be transmitted over a data path including Ethernet LAN segments and internet segments, then each hub or node needs to have a protocol stack that includes the several software layers needed to do SNMP processing, UDP filtering and routing, UDP/IP protocol processing and interface to the Ethernet LAN segment to which the node or hub is connected. The xe2x80x9cmaster hubxe2x80x9d is a hub or switch with a microprocessor in it that executes the proxy agent process and the subchannel communication protocol and which initiates automated topology discovery and routing table CRC calculations and carries out processing based upon the routing table CRC to insure that get and set or other management operations are based upon the correct network topology.
Each species in the main genus solves this main problem by having two shared structural and/or operational characteristics. First, every species in the main genus eliminates the need for separate data paths for the management and control data which are different from the high speed data paths. This is accomplished through use of an out-of-band subchannel data path through the same media used by the high speed data. The second component of the cost of ownership problem is solved by every species in the genus by elimination of the protocol stack at every node or hub and substituting a single protocol stack at one central location in embodiments where the network management process is physically separated from the network to be managed and coupled thereto through a wide area and/or local area net. This is accomplished in every species of the main genus through use of a network management process that talks to objects in the network to be managed through a proxy agent running on the machine having the single protocol stack. The use of a proxy agent allows the network management software to talk to a single entity, the proxy agent having a single IP address. The network management process does not have to be able to communicate with all the objects to be managed by whatever communication protocol is used to talk to them. The proxy agent does that. A simple communication protocol using the subchannel provides out-of-band communication between the proxy agent and the slave nodes. This structure having a subchannel management data path and a proxy agent eliminates the need for an IP address for every hub or node and eliminates the need for a protocol stack at every hub or node. In species where SNMP management protocol is used, it also eliminates the need for an SNMP (or other management protocol) license for every hub or node.
Every species in the main genus transmits out-of-band management packets which are transmitted using the same media as the high speed data, but wherein the management and control data is transmitted in subchannel packets and not encapsulated in any high speed data packets that would waste bandwidth needed for payload data. Because management data packets are usually not as large as the capacity of the high speed data blocks or packets, by not including the management packets in otherwise empty high speed data blocks, wasted bits space is avoided. This prevents reducing the throughput of the high speed data by in-band management packet transmission. The transmission of management and control data occurs over a subchannel portion of the Fourier spectrum of the high speed data where not very much energy from the high speed data is present to interfere with the subchannel transmissions using a simple communication protocol involving a single subchannel transceiver in every node, a routing table and routing instructions in subchannel packets. The species of this main genus using subchannel communications out-of-band on the main network media can be used on any network which has DC balanced encoding to suppress energy content at low frequencies from DC up to some frequency below the main energy content of the high speed data Networks using 8b/10b encoding protocol are examples of networks in which the invention can be used.
The communication protocol used by the proxy agent to transmit the management packets over the subchannel can be any known communication protocol. The management packets can be sent on the subchannel using either complex peer-to-peer communication protocols or simpler point-to-point communication protocols which are not peer-to-peer. All that is necessary is that the management subchannel packets get to the correct objects to be managed eventually. The details of how many transceivers are used in each hub or node to do the subchannel communication or the communication protocols they use to transmit on the subchannel are not important for species within this broadly defined genus. The subchannel communication protocol of the preferred embodiment can drop a packet, but this is not fatal since the proxy agent will retry although there is no retry between slaves.
The advantage of the species in this main genus is that the complexity, expense and inconvenience of using separate data paths for management data and the complexity and expense of a protocol stack and possible SNMP license at every node or hub are eliminated while providing the flexibility, reliability and expansion capability of a managed network. Such managed networks include multiple nodes or hubs each of which includes at least one port capable of bidirectional communication, but most of which contain multiple ports each of which must be capable of bidirectional communication.
A subgenus within this main genus solves a second problem. That problem is how to reduce the cost and complexity of the hardware and software for subchannel communication of management data. This problem is solved in all species of the subgenus by using a single subchannel transceiver in every hub and node with a multiplexer that switches the transceiver transmitter output and receiver input simultaneously between the multiple ports in accordance with a light weight, synchronous communication protocol for management or subchannel packets.
The species in this subgenus implement a simple, non peer-to-peer communication protocol by use of foreknowledge of the topology of the network so that routing instructions can be given to management packets. Simple parent-child relationships are used to define the topology and only one subchannel transceiver is used in each such node or hub regardless of how many ports each node or hub has. Simple rules are used to control the switching of the multiplexer so as to guide the single transceiver as to which port to listen to or transmit from. A simple slave software process controls switching by the multiplexer. The term xe2x80x9cslavexe2x80x9d is used herein to mean xe2x80x9cproxied devicexe2x80x9d which the proxy agent process talks to. Nodes and slaves are intended to be understand as meaning the same thing when speaking of nodes on the network other than the master hub node or the proxy agent process. The slave process controls the multiplexer to always listen to the port connected to the parent (the uplink port) until the parent sends a management packet and instructs the child slave to transmit it out a specific port to another slave. The incoming packet is referred to in the claims as a subchannel packet and is called a GUMP packet in the detailed description below of the preferred embodiment. GUMP packets are particular kinds of subchannel packets which include routing instructions in the header thereof based upon the topology of the network so as to instruct each slave through which the GUMP packets passes how to switch its multiplexer to make the relay. The protocol is synchronous, so a slave through which a GUMP packet passes keeps its multiplexer switched to the downlink port out which the packet was transmitted until a reply is received or timeout occurs. When either event occurs, the slave process switches the multiplexer back to the uplink port, and, if a reply was received, sends the reply packet to its parent. This process continues all the way up the chain. This simple protocol has no lost packets, no collisions since the master talks to only one slave at a time, and does not require complicated back-off and retry algorithms.
The foreknowledge of the topology necessary to provide these routing instructions in this subgenus may be provided by configuration data which is supplied by the network administrator or by automatic topology discovery. The automatic topology discovery process is practiced in the preferred embodiment. Automatic topology discovery has the advantage of eliminating the need for the network administrator to update the configuration data of all the slaves when the topology of the network changes.
One alternative embodiment employs one subchannel transceiver for every port so that broadcast subchannel communication protocols can be implemented.
Another alternative embodiment employs one subchannel transceiver in every port but multiple multiplexers, one for the transmit output and the other for the receiver input.